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INTERACTIVE SYSTEM FOR LOCAL 

INTERVENTION INSIDE A 
NONHUMOGENEOUS STRUCTURE 

The invention relates to an interactive system for local 5 
intervention inside a region of a nonhomogeneous structure. 

The performing of local interventions inside a nonho- 
mogeneous structure, such as intracranial surgical opera- 
tions or orthopedic surgery currently poses the problem of 
optimizing the intervention path or paths so as to secure, on 
the one hand, total intervention over the region or structure 
of interest, such as a tumor to be treated or explored and, on 
the other hand, minimal lesion to the regions neighboring or 
adjoining the region of interest, this entailing the localizing 
and then the selecting of the regions of the nonhomogeneous 
structure which are least sensitive to being traversed or the 15 
least susceptible to damage as regards the integrity of the 
structure. 

Numerous works aimed at providing a solution to the 
abovementioned problem have hitherto been the subject of 
publications. Among the latter may be cited the article 20 
entitled "Three Dimensional Digitizer (Neuronavigator): 
New Equipment for computed Tomography Guided Stereo- 
taxic Surgery", published by Eiju Watanabe, M.D., Takashi 
Watanabe, M.D., Shinya Manaka, M.D., Yoshiaki 
Mayanagi, M.D., and Kintomo Takakura, M.D. Department 25 
of Neurosurgery, Faculty of Medicine, University of Tokyo, 
Japan, in the journal Surgery Neurol. 1987: 27 pp. 543-547, 
by Elsevier Science Publishing Co., Inc. The Patent WO-A- 
88 09151 teaches a similar item of equipment. 

In the abovementioned publications are described in 30 
particular a system and an operational mode on the basis of 
which a three-dimensional position marking system, of the 
probe type, makes it possible to mark the three-dimensional 
position coordinates of a nonhomogeneous structure, such as 
the head of a patient having to undergo a neurosurgical 35 
intervention, and then to put into correspondence as a 
function of the relative position of the nonhomoc.eneous 
structure a series of corresponding images consisting of 
two-dimensional images sectioned along an arbitrary 
direction, and obtained previously with the aid of a medical 40 
imaging method of the "scanner" type. 

The system and the operational mode mentioned above 
offer a sure advantage for the intervening surgeon since the 
latter has available, during the intervention, apart from a 
direct view of the intervention, at least one two-dimensional 45 
sectional view enabling him to be aware, in the sectional 
plane, of the state of performance of the intervention. 

However, and by virtue of the very design of the system 
and of the operational mode mentioned above, the latter 
allow neither a precise representation of the state of perfor- 50 
mance of the intervention, nor partially or totally automated 
conduct of the intervention in accordance with a program for 
advance of the instrument determined prior to the interven- 
tion. 

Such a system and such an operational mode cannot 55 
therefore claim to eradicate all man-made risk, since the 
intervention is still conducted by the surgeon alone. 

The objective of the present invention is to remedy the 
whole of the problem cited earlier, and in particular to 
propose a system permitting as exact as possible a 60 
correlation, at any instant, between an intervention modeling 
on the screen and the actual intervention, and furthermore 
the representation from one or more viewing angles, and if 
appropriate in one or more sectional planes, of the nonho- 
mogeneous structure, the sectional plane or planes possibly 65 
being for example perpendicular to the direction of the path 
of advance of the instrument or of the intervention tool. 



675 

2 

Another objective of the present invention is also the 
implementation of a system penrritting simulation of an 
optimal trajectory of advance of the tool, so as to constitute 
an assisted or fully programed intervention. 

Finally, an objective of the present invention is to pro- 
pose a system making it possible, on the basis of the 
simulated trajectory and of the programed intervention, to 
steer the movement of the instrument or tool to the said 
trajectory so as to carry out the programed intervention. 

The invention proposes to this effect an interactive 
system for local intervention inside a region of a nonhomo- 
geneous structure to which is tied a reference structure 
containing a plurality of base points, characterized in that it 
comprises: 

means of dynamic display by three-dimensional imaging 
of a representation of the nonhomogeneous structure 
and of a reference structure tied to the nonhomoge- 
neous structure, including images of the base points, 
means of delivering the coordinates of the images of the 

base points in the first reference frame, 
means of securing the position of the non-homogeneous 
structure and the reference structure with respect to a 
second reference frame, 
marker means for delivering the coordinates of the base 

points in the second reference frame, 
means of intervention comprising an active member 
whose position is determined with respect to the second 
reference frame, 
means of optimizing the transfer of reference frames from 
the first reference frame to the second reference frame 
and vice versa, on the basis of the coordinates of the 
images of the base points in the first reference frame 
and of the coordinates of the base points in the second 
reference frame, in such a way as to reduce to a 
minimum the deviations between the coordinates of the 
images of the base points in the first reference frame 
and the coordinates of the base points, expressed in the 
said first reference frame with the aid of the said 
reference frame transfer tools, 
means for defining with respect to the first reference 
frame a simulated origin of intervention and a simu- 
lated direction of intervention, and 
reference frame transfer means using the said reference 
frame transfer tools to establish a bidirectional coupling 
between the simulated origin of intervention and the 
simulated direction of intervention and the position of 
the active member. 
A more detailed description of the system of the invention 
will be given below with reference to the drawings in which: 
FIG. 1 represents a general view of an interactive system 
for local intervention inside a region of a nonhomogeneous 
structure according to the present invention, 

FIG. 2 represents, in the case where the nonhomogeneous 
Structure consists of the head of a patient, and with a view 
to a neurosurgical intervention, a reference structure tied to 
the nonhomogeneous structure and enabling a correlation to 
be established between a "patient" reference frame and a 
reference frame of images of the patient which were made 
and stored previously, 

FIG. 3 represents an advantageous embodiment of the 
spacial distribution of the reference structure of FIG. 2, 

FIG. 4 preesents an advantageous embodiment of the 
intervention means set up on an operating table in the case 
of a neurosirgical intervention, 

FIGS. 5a and 56 represent a general flow diagram of 
functional steps implemented by the system, 
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FIGS. 6 thru 8 represent flow diagrams of programs 
permitting implementation of certain functional steps of 
FIG. 5b, 

FIG. 9a represents a flow diagram of a program permit- 
ting implementation of a functional step of FIG. 5a, 

FIG. 9b represents a flow diagram of a program permit- 
ting implementation of another functional step of FIG. 5a, 

FIGS. 10a and 10£ represent a general flow diagram of 
the successive steps of an interactive dialogue between the 
system of the present invention and the intervening surgeon 
and 

FIG. 10c represents a general flow diagram of the suc- 
cessive functional steps carried out by the system of the 
invention, having (sic) the intervention, prior to the 
intervention, during the intervention and after the interven- 
tion. 

The interactive system for local intervention according to 
the invention will firstly be described in connection with 
FIG. 1. 

Anonhomogeneous structure, denoted SNH, on which an 
intervention is to be performed, consists for example of the 
head of a patient in which a neurosurgical intervention is to 
be performed. It is however understood that the system of 
the invention can be used to carry out any type of interven- 
tion in any type of nonhomogeneous structure inside which 
structural and/or functional elements or units may be in 
evidence and whose integrity, during the intervention, is to 
be respected as far as possible. 

The system comprises means, denoted 1, of dynamic 
display by three-dimensional imaging, with respect to a first 
reference frame R 1? of a representation (denoted RSR) of a 
reference structure SR (described later) tied to the structure 
SNH, and a representation or modeling of the nonhomoge- 
neous structure, denoted RSNH. 

More precisely, the means 1 make it possible to display a 
plurality of successive three-dimensional images, from dif- 
ferent angles, of the representations RSNH and RSR. 

The system of the invention also comprises means, 
denoted 2, of tied positioning, with respect to a second 
reference frame R 2 , of the structures SNH and SR. 

In the present non-limiting example, the head of the 
patient, bearing the reference structure SR, is fixed on an 
operating table TO to which are fixed the means 2 of tied 
positioning. 

Of course, the patient whose head has been placed in the 
means 2 for tied positioning has previously been subjected 
to the customary preparations, in order to enable him to 
undergo the intervention. 

The means 2 of the tied positioning with respect to R 2 will 
not be described in detail since they can consist of any 
means (such as a retaining headset) normally used in the 
field of surgery or neurosurgery. The reference frame R2 can 
arbitrarily be defined as a tri-rectangular reference trihedron 
tied to the operating table TO, as represented in FIG. 1. 

Means 3 of marking, with respect to the second reference 
frame R^, the coordinates, denoted X2, Y2, Z2, of arbitrary 
points, and in particular of a certain number of base points 
of the reference structure SR are furthermore provided. 

These base points constituting the reference structure SR 
can consist of certain notable points and/or of marks fixed to 
the patient, at positions selected by the surgeon and in 
particular at these notable points. 

The system of the invention further comprises computing 
means 4 receiving means 3 of marking the coordinates X2, 
X2, 72, 

The computing means 4, as will be seen in detail later, are 
designed to elaborate optimal tools for reference frame 
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transfer using on the one hand the coordinates in R 2 , 
measured by the probe 3, of a plurality of base points of the 
structure SR, and on the other hand the coordinates in R x , 
determined by graphical tools of the computer M01 
5 (pointing by mouse, etc.), of the images of the corresponding 
base points in the representation RSR, so as to secure the 
best possible correlation between the information modeled 
in the computer equipment and the corresponding real- world 
information. 

10 There is furthermore provision for reference frame trans- 
fer means 11 designed to use the tools thus elaborated and 
to secure this correlation in real time. 

Moreover, means 40 are provided, as will be seen in detail 
later, for determining or modeling a reference origin of 

15 intervention ORI and a direction of intervention A. 

With the aid of the means 11, the modeled direction of 
intervention A can, at least prior to the intervention and at the 
start of the intervention, be materialized through an optical 
sighting system available to the surgeon, it being possible to 

20 steer this sighting system positionally with respect to the 
second reference frame R 2 . 

The sighting system will be described later. 
The system of the present invention finally comprises 
means 5 of intervention comprising an active member, 

25 denoted 50, whose position is specified with respect to the 
second reference frame R 2 . The active member can consist 
of the various tools used in surgical intervention. For 
example, in the case of an intercranial neurosurgical 
intervention, the active member could be a trephining tool, 

30 a needle, a laser or radioscope emission head, or an endo- 
scopic viewing system. 

According to an advantageous characteristic of the 
invention, by virtue of the reference frame transfer means 
11, the position of the active member can be controlled 

35 dynamically on the basis of the prior modeling of the origin 
of intervention ORI and of the direction of intervention A. 

The means 1 of dynamic display by three-dimensional 
imaging of the representations RSNH and RSR comprise a 
file 10 of two-dimensional image data. The file 10 consists 

40 for example of digitized data from tomographic sections, 
from radiographs, from maps of the patient's head, and 
contained in an appropriate mass memory. 

The successive tomographic sections can be produced 
prior to the intervention in a conventional manner, after the 

45 reference structure SR has been put in place on the nonho- 
mogeneous structure SNH. 

According to an advantageous feature, the reference 
structure SR can consist of a plurality of marks or notable 
points which can be both sensed by the marker means 3 and 

50 detected on the two-dimensional images obtained. 

Of course, the abovementioned two-dimensional tomo- 
graphic sections can likewise be produced by any medical 
imaging means such as a nuclear magnetic resonance sys- 
tem. 

55 In a characteristic and well-known manner, each two- 
dimensional image corresponding to a tomographic scanner 
section corresponds to a structural slice thickness of about 2 
to 3 mm, the pixels or image elements in the plane of the 
tomographic section being obtained with a precision of the 

60 order of ±1 mm. It is therefore understood that the marks or 
points constituting the reference structure SR appear on the 
images with a positional uncertainty, and an important 
feature of the invention will consist in minimizing these 
uncertainties as will be described later. 

65 The system also comprises first means HO for calculating 
and reconstructing three-dimensional images from the data 
from the file 10. 
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It also comprises a high-resolution screen 12 permitting determine the coordinates of the said marks in the reference 

the displaying of one or more three-dimensional or two- frame R 2 or in a reference frame tied to the latter, 

dimensional images constituting so many representations of It is important to note here that the general function of the 

the reference structure RSR and of the nonhomogeneous base points of the reference structure is, on the one hand, to 

structure SNH. 5 be individuaHy localizable on the reference structure, in 

Advantageously, the calculating means 110, the high- order to deduce from this the coordinates in R 2 , and on the 

resolution screen and the mass memory containing the file other hand, to be visualizable on the two-dimensional 

10 form part of a computer of the workstation type with images so as to be identified (by their coordinates in R 2 ) and 

conventional design and denoted MOl. included in the representation RSR on the screen. 

Preferably, the first calculating means 110 can consist of 10 It can therefore involve special marks affixed at arbitrary 

a CAD type program installed in the workstation MOl. points of the lateral surface of the structure SNH, or else at 

By way of non-limiting example, this program can be notable points of the latter, or else, when the notable points 

derived from the software marketed under the tradename can in themselves be localized with high precision both on 

"AUTOCAD" by the "AUTODESK" company in the the structure SNH and on the 2D sections, notable points 

United States of America. 15 totally devoid of marks. 

^ Such software makes it possible, from the various digi- In FIG. 2 a plurality of marks, denoted Ml to Mi, these 

tized two-dimensional images, to reconstruct threedimen- marks, in the case where the nonhomogeneous structure 

sional images constituting the representations of the struc- consists of the head of a patient, being localized for example 

aires RSR and RSNH in arbitrary orientations. between the eyebrows of the patient, on the latter's temples, 

Thus, as has furthermore been represented in FIG. 1, the 20 and at the apex of the skull at a notable point such as the 

calculating means 4 and 11 can consist of a third computer, frontal median point. 

denoted M02 in FIG. 1. More generally, for a substantially ovoid volume consti- 

The first and second computers MOl and M02 are tuting the nonhomogeneous structure, there is advanta- 

interconnected by a conventional digital link (bus or geously provision for four base points at least on the outer 

network). 25 surface of the volume. 

As a variant, the computers MOl and M02 can be Thus, as has been represented in FIG. 3, the four marks 

replaced by a single workstation. Ml to M4 of the reference structure are distributed so as 

The marker means 3 consist of a three-dimensional probe preferably to define a more or less symmetric tetrahedron, 

equipped with a tactile tip 30. The symmetry of the tetrahedron, represented in FIG. 3, is 

This type of three-dimensional probe, known per se and 30 materialized by the vertical symmetry plane PV and the 

not described in detail, consists of a plurality of hinged arms, horizontal symmetry plane PH. 

marked in terms of position with respect to a base integral According to an advantageous characteristic, as will be 

with the operating table TO. It makes it possible to ascertain seen later, the means of elaborating the reference frame 

the coordinates of the tactile tip 30 with respect to the origin transfer tools are designed to select three points of the 

0 2 of the reference frame R^, with a precision better than 1 35 tetrahedron which will define the "best plane" for the 

aim. reference frame transfer. 

The probe is for example equipped with resolvers deliv- Also, the presence of four or more points enables the 

ering signals representing the instantaneous position of the additional points) to validate a specified selection, 

abovementioned tactile tip 30. The resolvers are themselves More precisely, the presence of a minimum of four base 

connected to the circuits for digital/analog conversion and 40 points on the reference structure makes it possible to search 

sampling of the values representing these signals, these for the minimum distortion between the points captured on 

sampling circuits being interconnected in conventional man- the patient by the marker means consisting for example of 

ner to the second computer M02 in order to supply it with the three-dimensional probe and the images of these points 

the coordinates X2, X2, 72 of the tactile tip 30. on the representation by three-dimensional imaging, the 

As a variant or additionally, and as represented 45 coordinates of which are calculated during processing. The 

diagrammatically, the marker means 3 can comprise a set of best plane of the tetrahedron described earlier, that is to say 

video cameras 31 and 32 (or else infrared cameras) enabling the plane for which the uncertainty in the coordinates of the 

pictures to be taken of the structures SNH and SR. points between the points actually captured by the three- 

The set of cameras can act as a stereoscopic system dimensional probe and the points of the representation of the 

permitting the positional plotting of the base points of the 50 reference structure RSR, is minimal, then becomes the 

reference structure SR, or of other points of the nonhomo- reference plane for the reference frame transfer. Thus, the 

geneous structure SNH, with respect to the second reference best correlation will be established between a modeled 

frame R 2 . The positional plotting can be done for example direction of intervention and a modeled origin of 

by appending a laser beam emission system making it intervention, on the one hand, and the action of the member 

possible to illuminate successively the points whose coor- 55 50. Preferably, the origin of intervention will be placed at the 

dinates are sought, appropriate software making it possible center of the region in which the intervention is to be carried 

to then determine the position of these points one by one out, that is to say a tumor observed or treated for example, 

with respect to R 2 . This software will not be described since Furthermore, it will be possible to take the noted residual 

it can consist of position and shape recognition software uncertainty into account in order to effect the representation 

normally available on the market 60 of the model and of the tools on the dynamic display means. 

According to another variant, the marker means 3 can A more detailed description of the means of intervention 

comprise a telemetry system. 5 will now be given in connection with FIG. 4. 

In this case, the marks of the structure SR can consist of Preferably, the means of intervention 5 comprise a car- 
small radiotransmitters implanted for example on the rel- riage 52 which is translationally mobile along the operating 
evant points of the patient's head and designed to be visible 65 table TO, for example on a rack, denoted 54, whilst being 
on the two-dimensional images, appropriate electromagnetic driven by a motor, not shown, itself controlled by the 
or optical sensors (not shown) being provided in order to computer M02 for example, via an appropriate link. This 
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movement system will not be described in detail since it 
corresponds to a conventional movement system available 
on the market. As a variant, the carriage 52 can be mobile 
over a distinct path separated from the operating table TO, 
or immobile with respect to the operating table and then 5 
constitute a support. 

The support carriage 52 comprises in the first place a 
sighting member OV, constituting the above-mentioned 
sighting system, which can consist of a binocular telescope. 

The sighting member OV enables the surgeon, prior to the 10 
actual intervention, or during the latter, to sight the pre- 
sumed position of the region in which the intervention is to 
be carried out. 

Furthermore, and in a non-limiting manner, with the 
sighting member OV can be associated a helium-neon laser 15 
emission system, denoted EL, making it possible to secure 
the aiming of a fine positioning or sighting laser beam on the 
structure SNH and in particular, as will be seen in detail 
later, to indicate to the surgeon the position of an entry point 
PE prior to the intervention, to enable the latter to open the 20 
skull at the appropriate location, and likewise to indicate to 
him what the direction of intervention will be. Additionally, 
the illuminating of the relevant point of the nonhomoge- 
neous structure or at the very least the lateral surface of the 
latter enables the video cameras 31 and 32 to cany out, if 25 
necessary, a positional plotting. 

Preferably, a system for measuring position by telemetry 
53 is provided to secure the precise measurement of the 
position of the support carriage 52 of the sighting member 
OV and of the laser emission system EL. During the 30 
operation, and in order to secure the intervention, the 
carriage 52 can be moved along the rack 54, the position of 
the carriage 52 being measured very precisely by means of 
the system 53. The telemetry system 53 is interconnected 
with the microcomputer M02 by an appropriate link. 35 

The means of intervention 5 can advantageously consist 
of a guide arm 51 for the active member 50. 

The guide arm 51 can advantageously consist of several 
hinged segments, each hinge being equipped with motors 
and resolvers making it possible to secure control of move- 40 
ment of the end of the support arm and the positional 
plotting of this same end and therefore of the active member 
50 according to six degrees of freedom with respect to the 
carriage 52. The six degrees of freedom comprise, of course, 
three translation^ degrees of freedom with respect to a 45 
reference frame tied to the carriage 52 and three rotational 
degrees of freedom along these same axes. 

Thus, the support arm 51 and the member 50 are marked 
in terms of instantaneous position with respect to the second 
reference frame R 2 , on the one hand by way of the positional 50 
plot of the mobile carriage 52 and, on the other hand, by way 
of the resolvers associated with each hinge of the support 
arm 51. 

In the case of an intracranial neurosurgical surgical 
intervention, the active member 50 can be removed and can 55 
consist of a trephining tool, a needle or radioactive or 
chemical implant, a laser or radioisotope emission head or 
an endoscopic viewing system. These various members will 
not be described since they correspond to instruments nor- 
mally used in neurosurgery. 60 

The materializing of the modeled direction of intervention 
can be effective by means of the laser emitter EL. This 
sighting being performed, the guide arm 51 can then be 
brought manually or in steered manner into superposition 
with the direction of intervention A. 65 

In the case of manual positioning, the resolvers associated 
with the sighting member OV and the laser emitter EL, if 
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appropriate, make it possible to record the path of the 
sighting direction, constituting in particular the actual direc- 
tion of intervention, on the representation of the nonhomo- 
geneous structure in the dynamic display means 1. 

Furthermore, as will be described later and in preferential 
manner, the intervening surgeon will be able firstly to define 
a simulated intervention path and steer thereto the move- 
ments of the active member 50 in the nonhomogeneous 
structure in order effectively to secure all or part of the 
intervention. 

In this case, the progress of the intervention tool 50 is then 
steered directly to the simulated path (data ORI, A) by 
involving the reference frame transfer means 11 in order to 
express the path in the reference frame R 2 . 

A more detailed description of the implementation of the 
operational mode of the system of the invention will now be 
described in connection with FIGS. 5a and 5b. 

According to FIG. 5a, the first step consists in obtaining 
and organizing in memory the two-dimensional image data 
(step 100). Firstly, the nonhomogeneous structure SNH is 
prepared. In the case of a neurosurgical intervention for 
example, this means that the patient's head can be equipped 
with marks constituting the base points of the reference 
structure SR. These marks can be produced by means of 
points consisting of a dye partially absorbing the X-rays, 
such as a radiopaque dye. 

The abovementioned marks are implanted by the surgeon 
on the patient's head at notable points of the latter [sic], and 
images can then be taken of the nonhomogeneous structure 
SNH by tomography for example, by means of an apparatus 
of the X-ray scanner type. 

This operation will not be described in detail since it 
corresponds to conventional operations in the field of medi- 
cal imaging. 

The two-dimensional image data obtained are then con- 
stituted as digitized data in the file 10, these data being 
themselves marked with respect to the reference frame R ± 
and making it possible, on demand, to restore the two- 
dimensional images onto the dynamic display means 1, 
these images representing superimposed sections of the 
nonhomogeneous structure SNH. 

From the digitized image data available to the surgeon, 
the latter then proceeds, as indicated at 101 in FIG. 5a, to 
select the structures of interest of the abovementioned 
images. 

The purpose of this step is to facilitate the work of the 
surgeon by forming three-dimensional images which con- 
tain only the contours of the elements of the structure which 
are essential for geometrical definition and real-time moni- 
toring of the progress of the intervention. 

In the case where the nonhomogeneous structure SNH 
consists of the head of a patient, an analysis of the two- 
dimensional image data makes it possible, from values of 
optical density of the corresponding image-points, straight- 
away to extract the contours of the skull, to check the 
distance scales, etc. 

Preferably, the abovementioned operations are performed 
on a rectangle of interest for a given two-dimensional image, 
this making it possible, by moving the rectangle of interest, 
to cover the whole of the image. 

The above analysis is performed by means of suitable 
software which thus makes it possible to extract and vec- 
torize the contours of the structures which will be modeled 
in the representations RSNH and RSR. 

The structures modeled in the case of a neurosurgical 
intervention are for example the skull, the cerebral 
ventricles, the tumor to be observed or treated, the falx 
cerebri, and the various functional regions. 
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According to a feature of the interactive system of the 
invention, the surgeon may have available a digitizing table 
or other graphics peripheral making it possible, for each 
displayed two-dimensional image, to rectify or complete the 
definition of the contour of a particular region of interest. 5 

It will be noted finally that by superimposing the extracted 
contours on the displayed two-dimensional image, the sur- 
geon will be able to validate the extractions carried out. 

The extracted contours are next processed by sampling 
points to obtain their coordinates in the reference frame R 1? 
it being possible to constitute these coordinates as an ASCII 10 
type file. This involves step 102 for generating the three- 
dimensional data base. 

This step is followed by a step 103 of reconstructing the 
three-dimensional model This step consists firstly, with the 
aid of the CAD type software, in carrying out on the basis 15 
of the contours of the structures of interest constituted as 
vectorized two-dimensional images an extrapolation 
between the various sectional planes. 

The abovementioned extrapolation is carried out prefer- 
ably by means of a "B-spline" type algorithm which seems 20 
best suited. This extrapolation transforms a discrete item of 
information, namely the successive sections obtained by 
means of the scanner analysis, into a continuous model 
permitting three-dimensional representation of the volume 
envelopes of the structures. 25 

It should be noted that the reconstruction of the volumes 
constituting the structures of interest introduces an approxi- 
mation related in particular to the spacing and non-zero 
thickness of the acquisition sections. An important charac- 
teristic of the invention, as explained in detail elsewhere, is 30 
on the one hand to minimize the resulting uncertainties in the 
patient-model correlation, and on the other hand to take into 
account the residual uncertainties. 

The CAD type software used possesses standard functions 
which enable the model to be manipulated in space by 35 
displaying it from different viewpoints through just a crite- 
rion defined by the surgeon (step 104). 

The software can also reconstruct sectional representation 
planes of the nonhomogeneous structure which differ from 
the planes of the images from the file 10, this making it 40 
possible in particular to develop knowledge enhancing the 
data for the representation by building up a neuro- 
anatomical map. 

The surgeon can next (step 105) determine a model of 
intervention strategy taking into account the modeled struc- 45 
tures of interest, by evaluating the distance and angle ratios 
on the two-and three-dimensional representations displayed. 

This intervention strategy will consist, in actual fact, on 
the one hand in localizing the tumor and in associating 
therewith a "target point", which will subsequently be able 50 
to substitute for the origin common to all the objects (real 
and images) treated by the system, and on the other hand in 
determining a simulated intervention path respecting to the 
maximum the integrity of the structures of interest. This step 
can be carried out "in the office", involving only the work- 55 
station. 

Once this operation is performed and prior to the 
intervention, the following phase consists in implementing 
the steps required to establish as exact as possible a corre- 
lation between the structure SNH (real world) and the 60 
representation RSNH (computer world). This involves steps 
106 to 109 of FIG. 5b. 

Firstly, as represented in FIG. 56 at step 107, marking of 
the base points of the reference structure SR with respect to 
the second reference frame is carried out with the aid of the 65 
marker means 3, by delivering to the system the coordinates 
X2, Y2, 72 of the said base points. 
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The following step 106 consists in identifying on the 
representations RSNH and RSR displayed on the screen the 
images of the base points which have just been marked. 
More precisely, with the aid of appropriate graphics 
peripherals, these representations (images) of the base points 
are selected one by one, the workstation supplying on each 
occasion (in this instance to the computer M02) the coor- 
dinates of these points represented in the reference frame Rj. 

Thus the computer M02 has available a first set of 
three-dimensional coordinates representing the position of 
the base points in R 2 , and a second set of three-dimensional 
coordinates representing the position of the representations 
of the base points in R^ 

According to an essential feature of the invention, these 
data will be used to elaborate at 108, 109, tools for reference 
frame transfer (from R 2 to R 2 and vice versa) by calling 
upon an intermediate reference frame determined from the 
base points and constituting an intermediate reference frame 
specific to the reconstructed model. 

More precisely, the intermediate reference frame is con- 
structed from three base points selected so that, in this 
reference frame, the coordinates of the other base points 
after transfer from R 2 and the coordinates of the represen- 
tations of these other base points after transfer from R x are 
expressed with the greatest consistency and minimum dis- 
tortion. 

When the step of elaborating the reference frame transfer 
tools is concluded, these tools can be used by he system to 
secure optimal coupling between the real world and the 
computer world (step 1110). 

Furthermore, according to a subsidiary feature of the 
present invention, the system can create on the display 
means a representation of the nonhomogeneous structure 
and of the intervention member which takes account of the 
deviations and distortions remaining after the "best" refer- 
ence frame transfer tools have been selected (residual 
uncertainties). More precisely, from these deviations can be 
deduced by the calculating means a standard error likely to 
appear in the mutual positioning between the representation 
of the nonhomogeneous structure and the representation of 
elements (tools, sighting axes, etc.) referenced on R 2 when 
using the reference frame transfer tools. This residual 
uncertainty, which may in practice be given substance 
through an error matrix, can be used for example to repre- 
sent certain contours (tool, structures of interest to be 
avoided during the intervention, etc.) with dimensions larger 
than those which would normally be represented starting 
from the three-dimensional data base or with the aid of 
coordinates marked in R 2 , the said larger dimensions being 
deduced from the "normal" dimensions by involving the 
error matrix. For example, if the member were represented 
normally, in transverse section, by a circle of diameter Dl, 
a circle of diameter D2>D1 can be represented in substance, 
with the difference D2-D1 deduced from the standard error 
value. In this way, when a direction of intervention will be 
selected making it possible to avoid traversing certain struc- 
tures of interest, the taking into account of an "enlarged" size 
of the intervention tool will eradicate any risk of the 
member, because of the abovementioned errors, accidently 
traversing these structures. 

Back at step 105, and as will be seen in more detail with 
reference to FIGS. 9a and 96, the reference origin of 
intervention ORI and the direction of intervention A, that is 
to say the simulated intervention path, can be determined 
according to various procedures. 

According to a first procedure, the trajectory can be 
defined from two points, namely an entry point PE (FIG. 3) 
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and a target point, that is to say substantially the center of the The sub-module 1003 is itself followed, as represented in 
structure of interest consisting of the tumor to be observed FIG. 6, by a fourth sub-module 1004 for creating a unit 
or treated. Initially, these two points are localized on the three-dimensional orthogonal matrix N, characteristic of a 
model represented on the screen. right-handed orthonormal basis comprising three unit vec- 

According to a second methodology, the trajectory can be « t -^ t t*, a , . A . - A if _ , _ 

determined fern the abovementioned target point and from f s ^ J k ^d w h « ch ls ^ to the second reference 
a direction which takes account of the types of structures of ^ am ^°- Wlng £ * e nonhomogeneous struc- 

interest and of their positions with a view to optimally £ re SNH 15 P° sltlonall y ted ™* ^pecl *> ^ reference 
respecting their integrity. ame * _^ _^ _^ 

After the abovementioned step 108, the surgeon can at The three unit vectors i ', j \ k' are defined by the 
step 1110 perform the actual intervention. relations: 

The intervention can advantageously be performed by 
steering the tool or active member over the simulated _^ _^ 

intervention path, determined in step 1110. J=de/\\de \\ 

As a variant, given that the support arm 51 for the active 
member, equipped with its resolvers, continuously delivers 15 _ / _ \ _ _> 
the coordinates in R 2 of the said active member to the # = [ J EDA£T|/]|iSA J iJ"|| 

system, it is also possible to perform the operation manually I I 

or semi-manually, by monitoring on the screen the position _^ V> I 

and motions of a representation of the tool and by comparing 7' - /' a 1* 

them with the simulated, displayed i mtervention path. 20 M ^toi above, to the extent that the base points of 

It will furthermore be noted that the .modeled direction of me reference structure can be mafked ^ R ^fa M fa 
intervention can be materialized with the aid of the laser precision, so their representation in the computer base R, is 
beam described earlier, the positioning of the latter (with mar ked with a certain margin of error given on the one hand 
respect to R 2 ) being likewise carried out by virtue of the the non-zero thickness (typically from 2 to 3 mm) of the 
reference frame transfer tools. 25 slices represented by the two-dimensional images from the 

Certain functional features of the system of the invention file 10, and on the other hand (in general to a lesser extent) 
will now be described in further detail with reference to the definition of each image element or pixel of a section. 
FIGS. 6, 7, 8, 9a and 9h According to the invention, once a pair of transfer matri- 

The module for elaborating the reference frame transfer ces M, N has been elaborated with selected points A, B, C, 
tools (steps 108, 109 of FIG. 5b) will firstly be described 30 D> E, F, it is sought to validate this selection by using one or 
with reference to FIG. 6. more additional base points; more precisely, for the or each 

This module comprises a first sub-module 1001 for additional base point, this point is marked in R 2 with the aid 
acquiring three points A, B, C, the images of the base points of the P robe 30 > the representation of this point is marked in 
of SR on the representation RSNH (the coordinates of these R i after selection on the screen, and then the matrices N and 
points being expressed in the computer reference frame R,), 35 M are a PP hed res P ectlve *y to the coordinates obtained, in 
by successive selections of these points on the representa- order to obtain their expressions in the bases ( i j \ k ') and 
tion Tothiseffect the surgeon is led, by means of a ? a^ (T y f) tivel If ^ expressions are m d 

m erf ace such as a mouse to point successively at the three V agreement ; th J e two ^ es can b / regarded tt a smgle 
selected points A, B, C. intermediate reference frame, this securing the exact as 

The module for preparing the transfer tools also com- 40 possible mathematical coupling between the computer ref- 
pnses a second sub-module, denoted 1002, for creating a erence frame R 2 tied to the model and the "real" reference 
unit three-dimensional orthogonal matrix M, this matrix frame R 2 tied to the patient. 

being characteristic of a right-handed orthonormal basis In practice, the module for elaborating the reference frame 
represented by three unit vectors T, 7, k , which define an transfer tools can be designed to perform steps 1001 to 1004 
intermediate reference frame tied to R^ 45 m succession on basic triples which differ on each occasion 

-► -x (for example, if four base points have been defined associ- 

The unit vectors 1 , j and k are given by the relations: ated ^ four representations in RSR, there are four pos- 
sible triples), in order to perform the validation step 1005 for 
-7 -» each of these selections and finally in order to choose the 

j =ab I\)ab\\ 50 tr *pj e £ Qr wri £ cn me jj est validation is obtained, that is to say 

/ \ for which the deviation between the abovementioned 

expressions is smallest. This triple defines the "best plane" 
mentioned elsewhere in the description, and results in the 
"best" transfer matrices M and N. 
55 As a variant, it will be possible for the selection of the best 
plane to be made at least in part by the surgeon by virtue of 
his experience. 

where || || designates the norm of the relevant vector. It should be noted that the reference frame transfer will 

In the above relations, the sign "A" designates the vector only be concluded by supplementing the matrix calculation 
product of the relevant vectors. 60 with the matrices M, N with a transfer of origin, so as to 

Similarly, the module for preparing the transfer tools create a new common origin for example at the center of the 
comprises a third sub-module, denoted 1003, for acquiring tumor to be observed or treated (point ORI). This transfer of 
three base points D, E, F, of the structure SR, these three origin is effected simply by appropriate subtraction of vec- 
points being those whose images on the model are the points tors on the one hand on the coordinates in R v and on the 
A, B, C respectively. For this purpose, the surgeon, for 65 other hand on the coordinates in R 2 . These vectors to be 
example by means of the tactile tip 30, successively senses subtracted are determined after localization of the center of 
these three points to obtain their coordinates in R 2 . the tumor on the representation. 
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Furthermore, the means described above for establishing 
the coupling between the patient's world and the model's 
world can also be used to couple to the model's world that 
of map data, also stored in the workstation and expressed in 
a different reference frame denoted R 3 . In this case, since 5 
these data contain no specific visible mark, the earlier 
described elaboration of matrices is performed by substitut- 
ing for these marks the positions of notable points of the 
patient's head. These may be temporal points, the frontal 
median point, the apex of the skull, the center of gravity of 
the orbits of the eyes, etc. 10 

The corresponding points of the model can be obtained 
either by selection by mouse or graphics tablet on the model, 
or by sensing on the patient himself and then using the 
transfer matrices. 

The above step of elaborating the reference frame transfer 15 
tools, conducted in practice by the calculating means 4, 
makes it possible subsequently to implement the reference 
frame transfer means (FIGS. 7 and 8). 

With reference to FIG. 7, the first transfer sub-module 201 
comprises a procedure denoted 2010 for aquiring the coor- 20 
dinates XM, YM, ZM, expressed in R 1? of the point to be 
transferred, by selecting on the representation. 

The procedure 2010 followed by a procedure 2011 for 
calculating the coordinates XP, YP, ZP (expressed in R 2 ) of 
the corresponding real point on the patient through the 2 s 
transformation: 

{XP, YP, ZP}=M*N" 1 *{XM, YM, ZM} where M * N -1 
represents the product of the matrix M and the inverse 
matrix N. 

The procedure 2011 is followed by a processing proce- 30 
dure 2012 utilizing the calculated coordinates XP, YP, ZP, 
for example to indicate the corresponding point on the 
surface of the structure SNH by means of the laser emission 
system EL, or again to secure the intervention at the relevant 
point with coordinates XP, YP, ZP (by steering the active 35 
member). 

Conversely, in order to secure a transfer from SNH to 
RSNH, the second sub-module 202 comprises (FIG. 8) a 
procedure denoted 2020 for acquiring on the structure SNH 
the coordinates XP, YP, ZP (expressed in Rs) of a point to be 40 
transferred. 

These coordinates can be obtained by means of the tactile 
tip 30 for example. The procedure 2020 is followed by a 
procedure 2021 for calculating the corresponding coordi- 
nates XM, YM, ZM in R ± through the transformation: 45 

{XM, YM, ZM}=N*M- a *{XP, YP, ZP} 

A procedure 2022 next makes it possible to effect the 
displaying of the point with coordinates XM, YM, ZM on 
the model or again of a straight line or of a plane passing 
through this point and furthermore meeting other criteria, so 

It will be noted here that the two sub-modules 201, 202 
can used [sic] by the surgeon at any moment for the purpose 
of checking the valid nature of the transfer tools; in 
particular, it is possible to check at any time that a real base 
point, with coordinates known both in R 2 and R a (for 55 
example a base point of SR or an arbitrary notable point of 
the structure SNH visible on the images), correctly relocates 
with respect to its image after transferring the coordinates in 
step 2011. 

In the event of an excessive difference, a new step of 60 
elaboration of the transfer tools is performed. 

Furthermore, the sub-modules 201, 202 can be designed 
to also integrate the taking into account of the residual 
uncertainty, as spoken of above, so as for example to 
represent on the screen a point sensed not in a pointwise 65 
manner, but in the form for example of a circle or a sphere 
representing the said uncertainty. 
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From a simulated intervention path, for example on the 
representation RSNH, or from any other straight line 
selected by the surgeon, the invention furthermore enables 
the model to be represented on the screen from a viewpoint 
corresponding to this straight line. Thus the third transfer 
subroutine comprises, as represented in FIGS. 9a and 9b, a 
first module 301 for visualizing the representation in a 
direction given by two points and a second module 302 for 
visualizing the representation in a direction given by an 
angle of elevation and an angle of azimuth. 

The first module 301 for visualizing the representation in 
a direction given by two points comprises a first sub-module 
denoted 3010 permitting acquisition of the two relevant 
points which will define the selected direction. Hie coordi- 
nates of these points are expressed in the reference frame R x , 
these points having either been acquired previously on the 
nonhomogeneous structure SNH for example by means of 
the tactile tip 30 and then subjected to the reference frame 
transfer, or chosen directly on the representation by means 
of the graphics interface of the "mouse" type. 

The first sub-module 3010 is followed by a second 
sub-module denoted 3011 permitting the creation of a unit, 
orthogonal three-dimensional matrix V characteristic of a 

right-handed orthonormal basis T", T"> k" the unit vectors 

i j k being determined through the relations: 

7"-^ a7" 

where "A" represents the vector product and symbolizes 
the scalar product. 

The sub-module 3011 is followed by a routine 3012 
making it possible to secure for all the points of the entities 
(structures of interest) of the three-dimensional data base of 
coordinates XW, YW, ZW in R I a conversion into the 

orthonormal basis ( i j ", k ") by the relation: 

{XV, YV, ZV}=V*{XW, YW, ZW} 

The subroutine 3013 is then followed by a subroutine 
3014 for displaying the plane i", j'\ the subroutines 3013 and 
3014 being called up for all the points, as symbolized by the 
arrow returning to block 3012 in FIG. 9a. 

When all the points have been processed, an output 
module 3015 permits return to a general module, which will 
be described later in the description. It is understood that this 
module enables two-dimensional images to be reconstructed 
in planes perpendicular to the direction defined by A and B. 

In the same way, the second module 302 (FIG. 9b) for 
visualizing the representation from a viewpoint given by an 
angle of elevation and an angle of azimuth comprises a first 
sub-module 3020 for acquiring the two angles in the repre- 
sentation frame of reference. 

The selection of the angles of elevation and of azimuth 
can be made by selecting from a predefined data base or by 
moving software cursers associated with each view or else 
by modification relative to a current direction, such as the 
modeled direction of intervention. The sub-module 3020 is 
itself followed by a second sub-module 3021 for creating a 
unit orthoganal three-dimensional matrix W characteristic of 

a right-handed orthonormal basis of unit vectors i "', j 

k m . They are defined by the relations: 
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7"' • 1"' » O; 

k '" * z "' = sin(azLtnutii) 

^ 5 

/ '" ■ y = cos(elevation); 
i ■ jc"' = sin(elevation) 

10 

~J ni = X 1 " a 7"' 

A routine 3022 is then called for all the points of the 
entities of the three-dimensional data base of coordinates 
XW, YW, ZW and enables a first sub-routine 3023 to be 
called permitting calculation of the coordinates of the rel- 15 

evant point in the right-handed orthonormal bases T" 1 , 
— > — »■ 

j k' n through the transformation: 

{XV, YV, ZV}=V*{XW, YW, ZW} 

The sub-routine 3023 is itself followed by a sub-routine 20 
3024 for displaying the plane i"', j r ", the two sub-routines 
3023 and 3024 then being called up for each point as 
symbolized by the return via the arrow to the block 3022 for 
calling the abovementioned routine. When all the points 
have been processed, an output sub-module 3025 permits a 25 
return to the general menu. 

Of course, all of the programs, sub-routines, modules, 
sub-modules and routines destroyed earlier are managed by 
a general "menu" type program so as to permit interactive 
driving of the system by screen dialogue with the interven- 30 
ing surgeon by specific screen pages. 

A more specific description of a general flow diagram 
illustrating this general program will now be given in 
connection with FIGS. 10a and 10b. 

Thus, in FIG. 10a has been represented in succession a 
screen page 4000 relating to the loading of data from the 35 
digitized file 10, followed by a screen page 4001 making it 
possible to secure the parameterizing of the grey scales of 
the display on the dynamic display means 1 and to calibrate 
the image, for example. 

The screen page 4001 is followed by a screen page 4002 40 
making it possible to effect the generation of a global view 
and then a step or screen page 4003 makes it possible to 
effect an automatic distribution of the sections on the screen 
of the workstation. 

A screen page 4004 makes it possible to effect a manual 45 
selection of sections and then a screen page 4005 makes it 
possible to effect the selection of the strategy (search for the 
entry points and for the possible directions of intervention, 
first localizing of the target (tumor . . . ) to be treated . . . ), 
as defined earlier, and to select the position and horizontal, 50 
sagittal and frontal distribution of the sections. 

A screen page 4006 also makes it possible to effect a 
display of the settings of a possible stereotaxic frame. 

It will be recalled that the reference structure SR advan- 
tageously replaces the stereotaxic frame formerly used to 55 
effect the marking of position inside the patient's skull. 

There may furthermore be provided a screen-page 4007 
for choosing strategic sections by three-dimensional 
viewing, on selection by the surgeon, and then at 4008 the 
aligning of the references of the peripherals (tool, sighting 60 
members, etc., with the aid of the probe 30. 

A screen page 4009 is also provided to effect the search 
for the base points on the patient with the aid of the said 
probe, following which the steps of construction of the 
reference frame transfer tools and of actual reference frame 65 
transfer are performed, preferably in a user-transparent 
manner 
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Another screen page 4010 is then provided, so as to effect 
the localizing of the target on the representation (for 
example a tumor to be observed or treated in the case of a 
neurosurgical intervention) in order subsequently to deter- 
mine a simulated intervention path. 

Then a new screen page 4011 makes it possible to effect 
the setting of the guides for the tool on the basis of this 
simulated path before opening up the skin and bone flaps on 
the patient's skull. 

Then a new localizing step 4012 makes it possible to 
check whether the position of the guides corresponds cor- 
rectly to the simulated intervention path. 

The screen page 4012 is followed by a so-called inter- 
vention screen page, the intervention being performed in 
accordance with step 1110 of FIG. 5b. 

A more detailed description of the interactive dialogue 
between the surgeon and the system during a surgical, and 
in particular a neurosurgical, intervention will follow with 
reference to FIG. 10c and to all of the preceding description. 

The steps of FIG. 10c are also integrated in the general 
program mentioned earlier; there are undertaken in succes- 
sion a first phase I (preparation of the intervention), then a 
second phase II, (prior to the actual intervention, the patient 
is placed in a condition for intervention, the reference 
structure SR being tied to the second reference frame R^, 
then a third phase III (intervention) and finally a post- 
intervention phase IV. 

With a view to preparing the intervention, the system 
requests the surgeon (step 5000) to choose the elementary 
structures of interest (for example bones of the skull, 
ventricles, vascular regions, the tumor to be explored or 
treated, and the images of the marks constituting in the first 
reference frame the representation RSR). 

The choice of the elementary structures of interest is made 
on the display of the tomographic images, for example, 
called up from the digitized file 10. 

The system next performs, at step 5001, a modeling of the 
structures of interest, as described earlier. Then, the nonho- 
mogeneous structure having been thus constituted as a 
three-dimensional model RSNH displayed on the screen, the 
intervening surgeon is then led to perform a simulation by 
three-dimensional imaging, at step 5002, with a view to 
defining the intervention path of the tool 50. 

During phase II the patient being placed in a condition for 
intervention and his head and the reference structure SR 
being tied to the second reference frame R^, the surgeon 
performs at step 5003 a search for the position of the marks 
Ml to M4 constituting base points of the reference structure 
in the second reference frame R 2 , and then during a step 
5004, performs a search for the position of the sighting 
systems, visualizing member OV, or of the tools and inter- 
vention instruments 50, still in the second reference frame 
R 2 , so as, if appropriate, to align these implements with 
respect to R 2 . 

The system then performs the validation of the 
intervention/patient spaces and representation by three- 
dimensional imaging in order to determine next the common 
origin of intervention ORL In other words, the matrix 
reference frame transfer described above is supplemented 
with the necessary origin translations (origins 01 and 02 
aligned on ORI). 

This operation is performed as described earlier. 

Phase III corresponds to the intervention, during which 
the system effects at step 5006 a permanent coupling in real 
time between the direction of aim of the active member 50, 
and/or of the direction of aim of the sighting member OV 
(and if appropriate of the laser beam), with the direction of 
aim (of observation) simulated by three-dimensional imag- 
ing on the display means 1, and vice versa. 
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